Make your Operating System (64-bit)

Before diving deep into building our own operating system, we need to understand a few basic technology and terminologies related to this topic.

Let's talk about some of the basics of Operating system. An Operating system, is a piece of software that controls the hardware components of the system be it a phone, laptop, or desktop. It is in charge of the communication between the software and the hardware.

A typical operating system consists of the following components they are:

1. The Bootloader - controls the boot process

2. The Kernel - the core of the system and manages the CPU, memory and peripheral devices

3. Daemons - background services

4. Networking - communication systems for sending and receiving data between systems

5. The Shell - software that allows manipulation of the device through commands

6. Graphical Server - the system that shows graphics on the screen

7. Desktop Environment - software or system used for user interaction

8. Applications - programs that perform user's tasks such as word processors, paint, etc

What is User space and Kernel space?

User Space - the user’s applications are carried out in the user-space, where they can reach a subset of the machine’s available resources via kernel system calls. By using the core services provided the kernel, a user-level application can be created like a game or office productivity software for example.

Kernel Space - the kernel is found in an elevated system state, which includes a protected memory space and full access to the device’s hardware. This system state and memory space is altogether referred to as kernel-space. Within kernel space the core access to the hardware and system services are managed and provided as a service to the rest of the system.

https://linuxhint.com/linux-kernel-tutorial-beginners/

Pre-requisites

• GNU / Linux / Windows Subsystem for Linux (preferred)

• Assembler

• GCC - GNU Compiler Collection

• Xorriso - A package that creates, loads, manipulates ISO 9660 filesystem images. (man xorriso)

• grub-mkrescue - Make a GRUB rescue image, this package internally calls the Xorisso functionality to build an ISO image

• QEMU - Quick EMUlator to boot our kernel in virtual machine without rebooting the host

Installation of Windows Subsystem for Linux

• Turn on the windows feature

  

• Download Ubuntu from the Microsoft Store

  

Download and Extract the Source code of GCC Cross-compiler and Binutils

• GCC Cross-compiler can be defined as a compiler which can convert instructions into machine code or low level code for a computer other than that on which it is run.

• Binutils are collections of binary tools.

  The GNU Binary Utilities, or binutils, are a set of programming tools for creating and managing binary programs, object files, libraries, profile data, and assembly source code. The main ones are: ld - the GNU linker as - the GNU assembler

  For more information read here:

  https://en.wikipedia.org/wiki/GNU_Binutils

  https://www.gnu.org/software/binutils/

  https://opensource.com/article/19/10/gnu-binutils

Download Binutils
$ wget http://ftp.gnu.org/gnu/binutils/binutils-2.35.tar.gz

Download GCC compiler
$ wget xvf gcc-10.2.0 http://ftp.gnu.org/gnu/gcc/gcc-10.2.0/gcc-10.2.0.tar.gz

Extract downloaded files to a folder using the following commands

$ tar xvf binutils-2.35.tar.gz
$ tar xvf gcc-10.2.0.tar.gz

Install some essential packages before we start the build process.

$ sudo apt-get install libmpc-dev
$ sudo apt-get install -y libcloog-isl-dev
$ sudo apt-get install libisl-dev
$ sudo apt-get install libmpfr-dev
$ sudo apt-get install libgmp3-dev

To install the new compilers in a HOME folder. Set the PATH variable as shown below (optional)

$ export PREFIX="$home/opt/cross"
$ export TARGET=i686-elf
$ export PATH ="$PREFIX/bin:$PATH"

Build and Install the Binutils

Install the new compiler in a HOME folder. This step builds and installs the cross-assembler, cross-linker, and other tools.

$ rm -rfv binutils-build #remove binutils - build if it already exists
$ mkdir binutils-build
$ cd binutils-build
$ ../binutils-2.35/configure --prefix=$home/opt/cross --target=i686-elf --disable-nls
$ make -j8 # 8 is size of RAM or make all or make -j4
$ sudo make install
$ cd ..

Build and Install the GCC Cross-Compiler

The GCC Cross-compiler, which is a compiler that builds programs for another machine. All you need is a Unix-like environment with a recent version of GCC already installed.

This step will build GCC’s C and C++ cross-compilers only, and install them to /opt/cross/bin. It won’t invoke those compilers to build any libraries just yet.

$ rm -rfv gcc-build
$ mkdir gcc-build
$ cd gcc-build
$ ../gcc-10.2.0/configure --prefix=$home/opt/cross --target=i686-elf --enable-languages=c,c++ --disable-nls --without-headers
$ make -j8 all-gcc #almost a day to complete if -j8 not used or use make all-gcc
$ make -j8 all-target-libgcc / make -j8 all-libgcc
$ sudo make install-gcc
$ sudo make install-target-libgcc

For more information read here:

https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/

https://www6.software.ibm.com/developerworks/education/l-cross/l-cross-ltr.pdf

https://tssurya.wordpress.com/2014/08/28/explanation-of-boot-s/

https://www.gnu.org/software/grub/manual/multiboot/multiboot.html

https://os.phil-opp.com/cross-compile-binutils/

http://www.ifp.illinois.edu/~nakazato/tips/xgcc.html#binutil

Essentials files for creating 64-Bit Operating System

boot.s - Kernel entry point that sets up the processor environment

kernel.c - Contains the actual kernel routines

linker.ld - used for linking the above files

Create a folder and add the following files:

boot.s

# Declare constants used for creating a multiboot header.
.set ALIGN,    1<<0             # align loaded modules on page boundaries
.set MEMINFO,  1<<1             # provide memory map
.set FLAGS,    ALIGN | MEMINFO  # this is the Multiboot 'flag' field
.set MAGIC,    0x1BADB002       # 'magic number' lets bootloader find the header
.set CHECKSUM, -(MAGIC + FLAGS) # checksum of above, to prove we are multiboot

#These lines describe the constants recognized by a multiboot compatible loader.

# Declare a header as in the Multiboot Standard.
.section .multiboot
.align 4
.long MAGIC 
.long FLAGS 
.long CHECKSUM

# Allocate room for a small temporary stack as a global variable called stack.
.section .bootstrap_stack
stack_bottom:
.skip 16384 # 16 KiB
stack_top:

# The linker script specifies _start as the entry point to the kernel and the
# bootloader will jump to this position once the kernel has been loaded.
.section .text
.global _start
.type _start, @function

_start:
	movl $stack_top, %esp
	call kernel_main
	cli

# Infinite loop 
hang:
	hlt
	jmp hang

Assemble the above boot.s file using the following command (optional, as a make file is created for assemble, linker and compilation)

$ ./i686-elf-as ~/OS/boot.s -o ~/OS/boot.o

kernel.c

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>

static const uint8_t COLOR_BLACK = 0;
static const uint8_t COLOR_BLUE = 1;
static const uint8_t COLOR_GREEN = 2;
static const uint8_t COLOR_CYAN = 3;
static const uint8_t COLOR_RED = 4;
static const uint8_t COLOR_MAGENTA = 5;
static const uint8_t COLOR_BROWN = 6;
static const uint8_t COLOR_LIGHT_GREY = 7;
static const uint8_t COLOR_DARK_GREY = 8;
static const uint8_t COLOR_LIGHT_BLUE = 9;
static const uint8_t COLOR_LIGHT_GREEN = 10;
static const uint8_t COLOR_LIGHT_CYAN = 11;
static const uint8_t COLOR_LIGHT_RED = 12;
static const uint8_t COLOR_LIGHT_MAGENTA = 13;
static const uint8_t COLOR_LIGHT_BROWN = 14;
static const uint8_t COLOR_WHITE = 15;

uint8_t make_color(uint8_t fg, uint8_t bg)
{
	return fg | bg << 4;
}

uint16_t make_vgaentry(char c, uint8_t color)
{
	uint16_t c16 = c;
	uint16_t color16 = color;
	return c16 | color16 << 8;
}

size_t strlen(const char* str)
{
	size_t ret = 0;
	while ( str[ret] != 0 )
		ret++;
	return ret;
}

static const size_t VGA_WIDTH = 80;
static const size_t VGA_HEIGHT = 24;

size_t terminal_row;
size_t terminal_column;
uint8_t terminal_color;
uint16_t* terminal_buffer;

void terminal_initialize()
{
	terminal_row = 0;
	terminal_column = 0;
	terminal_color = make_color(COLOR_LIGHT_GREY, COLOR_BLACK);
	terminal_buffer = (uint16_t*) 0xB8000;
	for ( size_t y = 0; y < VGA_HEIGHT; y++ )
		for ( size_t x = 0; x < VGA_WIDTH; x++ )
		{
			const size_t index = y * VGA_WIDTH + x;
			terminal_buffer[index] = make_vgaentry(' ', terminal_color);
		}
}

void terminal_setcolor(uint8_t color)
{
	terminal_color = color;
}

void terminal_putentryat(char c, uint8_t color, size_t x, size_t y)
{
	const size_t index = y * VGA_WIDTH + x;
	terminal_buffer[index] = make_vgaentry(c, color);
}

void terminal_putchar(char c)
{
	terminal_putentryat(c, terminal_color, terminal_column, terminal_row);
	if ( ++terminal_column == VGA_WIDTH )
	{
		terminal_column = 0;
		if ( ++terminal_row == VGA_HEIGHT )
		{
			terminal_row = 0;
		}
	}
}

void terminal_writestring(const char* data)
{
	size_t datalen = strlen(data);
	for ( size_t i = 0; i < datalen; i++ )
		terminal_putchar(data[i]);
}

void kernel_main()
{
	terminal_initialize();
	terminal_writestring("Hello, kernel World!\n");
}

Compile this kernel.c file using the following command

$ ./i686-elf-gcc -c kernel.c -o kernel.o -std=gnu99 -ffreestanding -O2 -Wall -Wextra

linker.ld

ENTRY(_start)

SECTIONS
{
	/* Begin putting sections at 1 MiB, a conventional place for kernels to be
	   loaded at by the bootloader. */
	. = 1M;

	/* First put the multiboot header, as it is required to be put very early
	   early in the image or the bootloader won't recognize the file format.
	   Next we'll put the .text section. */
	.text BLOCK(4K) : ALIGN(4K)
	{
		*(.multiboot)
		*(.text)
	}

	/* Read-only data. */
	.rodata BLOCK(4K) : ALIGN(4K)
	{
		*(.rodata)
	}

	/* Read-write data (initialized) */
	.data BLOCK(4K) : ALIGN(4K)
	{
		*(.data)
	}

	/* Read-write data (uninitialized) and stack */
	.bss BLOCK(4K) : ALIGN(4K)
	{
		*(COMMON)
		*(.bss)
		*(.bootstrap_stack)
	}
}

Link the kernel and boot object files using the following command

$ ./i686-elf-gcc -T linker.ld -o myos.bin -ffreestanding -O2 -nostdlib boot.o kernel.o -lgcc

Building a Bootable ISO Image

After the above steps are executed successfully without any error, we can now make a bootable ISO image that will run on virtual machines such as virtualbox, qemu, vmware etc. We have to create a bootable image containing the GRUB bootloader and our kernel using the program grub-mkrescue. You will need to install the GRUB utility programs and the program Xorriso (latest version) to create the ISO file.

First, we need to create a file called grub.cfg:

menuentry "myos" {
multiboot /boot/myos.bin
}

The steps for making the ISO image is the following (optional, as make file is created):

$ mkdir -p isodir
$ mkdir -p isodir/boot
$ cp myos.bin isodir/boot/myos.bin
$ mkdir -p isodir/boot/grub
$ cp grub.cfg isodir/boot/grub/grub.cfg
$ grub-mkrescue -o myos.iso isodir

Testing the new Operating System

Use Virtualbox or Qemu to run the image file.

$ qemu-system-i386 -cdrom myos.iso
or
>qemu-system-i386.exe -cdrom <path-to-ISO-file>myos.iso

Make File creation

To simplify the whole process of building the operating systems and execute each stage step by step, we have to create a Make file. Use the code below:

AS:=/opt/cross/bin/i686-elf-as
CC:=/opt/cross/bin/i686-elf-gcc

CFLAGS:=-ffreestanding -O2 -Wall -Wextra -nostdlib -nostartfiles -nodefaultlibs
CPPFLAGS:=
LIBS:=-lgcc

OBJS:=\
boot.o \
kernel.o \

all: myos.bin

.PHONEY: all clean iso run-qemu

myos.bin: $(OBJS) linker.ld
	$(CC) -T linker.ld -o $@ $(CFLAGS) $(OBJS) $(LIBS)

%.o: %.c
	$(CC) -c $< -o $@ -std=gnu99 $(CFLAGS) $(CPPFLAGS)

%.o: %.s
	$(AS) $< -o $@

clean:
	rm -rf isodir
	rm -f myos.bin myos.iso $(OBJS)

iso: myos.iso

isodir isodir/boot isodir/boot/grub:
	mkdir -p $@

isodir/boot/myos.bin: myos.bin isodir/boot
	cp $< $@

isodir/boot/grub/grub.cfg: grub.cfg isodir/boot/grub
	cp $< $@

myos.iso: isodir/boot/myos.bin isodir/boot/grub/grub.cfg
	grub-mkrescue -o $@ isodir

run-qemu: myos.iso
	qemu-system-i386 -cdrom myos.iso

Run the following commands to execute make

$ make all
$ make iso

Final Outputs

After running the ISO image file we will get a grub menu to select the new operating system. This is a minimal OS and displays the message "Hello, kernel World!"